\documentclass{article}

\usepackage[pdftex]{graphicx}
\usepackage{float}
\usepackage{url}
\DeclareGraphicsExtensions{.png,.jpg}

\author{James Babcock, <jab299@cornell.edu>\\
Cornell University\\
Under Prof. Andrew Myers}
\title{Guishell}
\begin{document}

\maketitle

\section{Abstract}

The Unix command line is used almost universally among programmers and system
administrators. It's powerful, it's a standard skill, and it's available on
every operating system. On the other hand, it has a steep learning curve, and
many anachronisms; it evolved incrementally all the way from teletypes to
modern PCs. Guishell modifies the traditional shell/terminal architecture to
add features which were previously impossible, by replacing the shell with its
own shell \texttt{guish} and the terminal with its own \texttt{guiterm}. These
features reduce the learning curve and make it convenient to do things that
were awkward or impossible before.

\section{Sample Session}

The best way to illustrate the differences between guishell and earlier shells
is through a sample session. Commands typed by the user are in \textbf{bold}.

\begin{quote}
	\lbrack someuser@mydesktop \~{}\rbrack\$ \textbf{ssh myserver} \\
	\texttt{Password:} \\
	\texttt{someshell-1.2.3\$} \textbf{guish} \\
	\lbrack someuser@myserver \~{}\rbrack\$ \textbf{cd \~{}/example} \\
	\lbrack someuser@myserver example\rbrack\$ \textbf{ls} \\
		% TODO: Put icons in this transcript (for files, directories, job action, and 'ls' view action
		\begin{tabular}{|l l l l l|}
			\hline
			\multicolumn{5}{|l|}{\lbrack\rbrack \lbrack\rbrack ls /home/someuser/example} \\
			.. & 1 hidden & foo.txt & foo.zip & readme.pdf \\
			\hline
		\end{tabular}
\end{quote}

In Guishell, `ls' is a builtin function, which creates a special widget for
listing files. This widget adds extra features beyond what the normal `ls'
program would do. First, at the top of the output, there is a toolbar with
buttons to change between regular and list (-l) view, and refresh. Next to it
is the command associated with the current view, canonicalized so it shows the
full path.

Double clicking on `1 hidden' will replace the `ls' command shown with `ls
-a', revealing files whose names start with a dot. Selecting files with the
mouse will add them to the command currently being typed. Right clicking on a
file or directory brings up a menu of common file management commands, such as
\texttt{rm}(1), while dragging files brings up a menu with \texttt{cp}(1)
(copy) and \texttt{mv}(1) (move). This is for the benefit of new users, to save
them from having to memorize command names; they can find them by brownsing
menus instead. In the future, the contents of these menus will be configurable
and filetype dependent, and more options will be added, so that auser could
use the file context menu to learn how to use programs like \texttt{chmod}(1)
and \texttt{tar}.

\begin{quote}
	\lbrack someuser@myserver example\rbrack\$ \textbf{vi foo.txt} \\
	\texttt{
		Note that the output of a running program, unlike the \\
		prompt, uses a monospace font, because some programs \\
		depend on it. This program uses escape sequences to move \\
		the cursor around, clear the screen, scroll, etc., but \\
		their effect is limited to this subwindow. \\
		\~{} \\
		\~{} \\
		\~{} \\
		\~{} \\
		\~{} \\
		\~{} \\
		\~{} \\
		\~{} \\
		:q
	} \\
	\lbrack someuser@myserver example\rbrack\$ \textbf{get foo.zip}
		% TODO: Put a progress bar here
\end{quote}

In guishell, both shell commands and (s)ftp-style commands work at the same
time. \texttt{get} and \texttt{put} are special builtins for transferring
files; \texttt{lcd} changes the directory files are downloaded to and uploaded
from.

While \texttt{get} and \texttt{put} are conceptually thought of as being run on
a server, they also affect the client. For security reasons, they can't be
treated in the same way as normal builtins; they can't be used inside scripts
or pipes, since that would allow a compromised host to retrieve and place files
on the client. Instead, they're client builtins; \texttt{guiterm} recognizes
when they're typed and handles them specially, rather than sending them to the
server to be executed.

\begin{quote}
	\lbrack someuser@myserver example\rbrack\$ \textbf{localview foo.pdf}
		% TODO: Put a progress bar here
\end{quote}

This downloads foo.pdf and opens it locally with an associated viewer,
such as Acrobat. Double clicking on foo.pdf runs this command, and adds it to
the history so the user can see the text equivalent of the command.

\begin{quote}
	\lbrack someuser@myserver example\rbrack\$ \textbf{cat foo.pdf}
\end{quote}

A bunch of random garbage goes here, because foo.pdf is a binary file, not
text. Because it's binary data that's not meant to be written to a terminal, it
may contain escape sequences that put the terminal in a strange state. With a
normal terminal, you'd put it back with the \texttt{reset} command. In
guishell, you don't have to, because terminal state is not shared between
commands.

\section{Architecture}

The Unix command line consists of a terminal, a shell, and programs run from
the shell. The terminal provides a user interface for the shell. It interacts
with it by reading and writing from a pseudoterminal (pty), which is a
bidirectional pipe with some extra options. Each pty has a master side, which
is a file descriptor used by the process which created it, and a slave side,
which is used as the stdin and stdout of other processes. Data written to
either side of a pseudoterminal is read by the other. In the usual arrangement,
there is one pty, created by the terminal, which the shell and all programs
run under it share, so only one program can write to the screen at a time or
else outputs will mix together. While waiting for a command to input, the shell
outputs to display a prompt and text typed so far; while the command is
running, that command produces output but the shell is mute; when that command
is stopped, the shell can output again until another command starts, etc.
However, if a command runs in the background, its output will be mixed in with
something else's, and so probably end up on the wrong part of the screen. With
guishell, on the other hand, every program gets its own pseudoterminal.

\begin{figure}[H]
	\begin{center}
		\includegraphics[scale=0.6]{ptydiagram}
		\caption{Processes and ptys for a traditional Unix shell (left) and
		Guishell (right). Pseudoterminals are color-coded to match their
		creator and master. Data flows over red edges using escape sequences,
		and over blue edges using Guishell protocol.}
	\end{center}
\end{figure}

In this arrangement, programs receive their input from and send their output to
the shell, rather than directly to the terminal. The shell is responsible for
passing the output along, but first, it labels it with a job ID, to indicate
which command was responsible for the output. The terminal is then responsible
for determining where on the screen it goes, which it does using sub-terminals.

This also sidesteps an old problem with programs that don't print a newline as
the last thing written. Compare what happens with a traditional shell:
\begin{quote}
\begin{verbatim}
	[user@host pwd]$ echo -n "Hello, world"
	Hello, world[user@host pwd]$ 
\end{verbatim}
\end{quote}
With guish:
\begin{quote}
\begin{verbatim}
	[user@host pwd]$ echo -n "Hello, world"
	Hello, world
	[user@host pwd]$ 
\end{verbatim}
\end{quote}
The problem for traditional shells is that the terminal can't tell which output
is from the program and which is the prompt from the shell, while the shell
can't tell whether the newline is missing or not because it can't read the
child program's output. Guishell sidesteps the issue entirely, by passing
program output through the shell and tagging its origin.

If one of the jobs is another instance of \texttt{guish}, then it becomes a
nested session. This most commonly occurs when using a program such as ssh to
connect to another host. To detect when this has occurred, the first thing
guish does when it runs is a handshake, which guiterm recognizes.

\begin{figure}[H]
	\begin{center}
		\includegraphics[scale=0.6]{sshdiagram}
		\caption{Nested shells over ssh, before and after the handshake. As
		before, red edges use VT100-style escape sequences and blue edges use
		Guishell's protocol.}
	\end{center}
\end{figure}

In principle, this handshake would include a fallback mechanism, so that if
\texttt{guish} is set as the login shell and someone tries to log in with
\texttt{xterm} or \texttt{PuTTY} instead, it will fall back on another shell.
This is planned, but not yet implemented, so it will instead fail; since other
shells can't speak Guishell's protocol, making \texttt{guish} your login shell
prevents logging in with any terminal besides \texttt{guiterm}.

\section{Protocols}

Guishell uses two protocols. Programs run from the shell produce escape
sequences to control the cursor and text formatting. These are passed as-is
from the program to the shell to the terminal, then rendered. The
implementation of escape sequence handling comes from PuTTY, a previously-
established terminal emulator. These escape sequences are theoretically
standardized in ECMA-48, but implementations differ greatly in practice.

Separate from the escape sequence protocol, the Guishell protocol is used for
communication between the shell (\texttt{guish}) and terminal
(\texttt{guiterm}). It does not support formatting text, but rather, provides
a remote procedure call interface between the client and server, with functions
for things like running commands on the server, setting up file transfers,
giving keyboard input to programs on the server and displaying program output
on the client. This protocol is documented, but is so far unique to Guishell.

\section{Scrollback History}

Terminals have a scroll bar, so you can refer back to the history of previous
commands and their output. However, sometimes a command outputs more than was
expected, such as thousands of pages of debug messages or an infinite number of
copies of the string ``y\\{}n'' from \texttt{yes}(1). Ordinarily, this would fill
the history up to its maximum size, displacing anything before it from the
scrollback history. This is because the terminal forgets the oldest text first,
but in Guishell, it works differently. Rather than a single, shared maximum
history size, Guishell limits the amount of scrollback that will be remembered
for any one command, so that when one program produces runaway output, others
won't be forced out of the history for it.

\section{Portability}

Guishell has a shell and terminal part, the shell being easily portable and
the terminal being hard. \texttt{guish} requires only POSIX and standard C++
functions; \texttt{guiterm} on the other hand requires the Trolltech Qt4
library, which may not be available and which has dependencies of its own. Qt
provides widgets for user interface, utility classes, a build system, and its
own dialect of C++. Fortunately, when accessing a host remotely, only
\texttt{guish} is needed, so systems that are only accessed remotely don't need
Qt.

\section{Limitations and Future Work}

A shell is a very large project, with many pieces of functionality which are
well-understood but nevertheless time consuming to implement. Much of this
functionality is missing. While there are new features focusing on the modified
shell/terminal interaction, the shell and terminal individually are quite
minimal. Guishell is not yet complete enough that anyone would want to use it
for day-to-day tasks, although it is close.

Guishell is completely unusable for scripting; it assumes it is always running
interactively, and is missing loops and branching, subshells, functions, and
most standard shell builtin variables. An unfortunate consequence of this is
that it can't run a profile or .guishrc file by itself, so it must inherit the
path and other environment variables from a parent shell, preventing its use as
a login shell.

Some common elements of shell syntax are missing, including the `$>$', `$>>$',
`$>\&$' and `$<$' redirection operators, most of the builtin variables,
aliasing, and arithmetic expansion (eg \texttt{\$((1+1))}). There is a command
history, but it is kept in the terminal, rather than the shell, and can't yet
be accessed with !startofcmd-style commands. There is tab completion, but it is
limited to filenames only, not context-sensitive, and doesn't search the path.

Since Guishell is an open source project, it is my hope that it will take off
and people will fill in the missing features they want most. Since the features
that are missing are ones for which there are already implementations to refer
to, this should be relatively straightforward.

In addition to filling in standard features which are missing, there are many
features which are possible and which would be useful, but which have never
been done before. One possibility is to allow HTML in program outputs. For
example, instead of \texttt{man} and \texttt{info} displaying straight,
monospaced text, they could be replaced with programs which display nicely
formatted, hyperlinked and clickable text. Most projects already have HTML
documentation available, of quality greater than or equal to that of the man
pages. Being able to output HTML could also enable Web developers to preview 
apps directly, without having to deploy them to a Web server; and console
applications which output HTML and receive form results as input, would be
immensely useful for beginning programmers who plan to later move to Web
development.

\section{Conclusion}

Changing the Unix command line's architecture, as Guishell does, makes a slew
of new and valuable features possible. Guishell proves that a better Unix
command line is possible, but only begins to scratch the surface. It is my
hope that this will open the floodgates for other shell and terminal
improvements.

\section{Getting Guishell}

Guishell is hosted on Google Code at \url{http://code.google.com/p/guishell},
and is licensed under the GNU General Public License.

\section{Acknowledgements}

Thanks to my project advisor, Andrew Myers, for supervising this project; to
Simon Tatham, author of PuTTY, for terminal emulation code; and to all those
who gave feedback and bug reports along the way.

\end{document}
